Bite-sized TDD plan for iter16: prep (fix 3 stale iter15 tests),
phased label changes (remove det_log_lines, add log_content, stack_trace,
diff_patch with regression-guard tests), audit tooling (build sample,
compute metrics), execution (build sample → 3-model annotate → report →
gate → iterate), iteration report + PR.

- Update label count to 40 (was 29, pre-iter15).
- Refresh expected label list to include iter15's 9 language labels.
- Invert routing-table coverage assertion: ROUTING_TABLE keys must be
  in DETECTION_LABELS, not the reverse. Sub_types without an entry
  fall through to the default model by design, and iter16 introduces
  detection-only semantic labels that will never have routing entries.
- Skip test_routing_uses_correct_api_keys when ROUTING_TABLE has no
  anthropic entries (iter15 migrated everything to openrouter).

log_lines remains a ContentSubType. The detection label was redundant
with the sub_type column and will be replaced by a proper cross-cutting
det_log_content in subsequent commits. No trained model depends on
det_log_lines (detection head has never been trained).

Documents which detection labels are NOT mirrored by a ContentSubType.
Empty in this commit — populated in subsequent phases with log_content,
stack_trace, and diff_patch.

First detection-only label — fires on embedded log output anywhere in a
row, distinct from sub_type=log_lines which means the row IS primarily
logs. Strict density+schema definition with uppercase-severity rule to
keep false positives bounded at the ~0.1% target class prevalence.

Two precision refinements to the log_content definition block from
code-quality review:

- JSON log records: clarify that a record may span multiple lines when
  pretty-printed and should be counted as ONE density unit regardless
  of its line count (resolves a contradiction with the top-level
  'consecutive lines' density rule).
- logfmt: require the timestamp/severity co-occurrence to be within
  the same line (via ts=/time=/level=/severity= keys) rather than the
  ambiguous 'co-occurring' wording which could be read as adjacent.

Multi-frame programmatic traceback detection across Python, Java/JVM,
.NET, JS/Node, Go, Rust, and Ruby. Requires two or more frames with
file:line or package.Class.method locators — single-line 'Error: foo'
messages and prose 'at line 5' do not qualify. Co-fires with log_content
when embedded in log output.

Three precision refinements from code-quality review:

- Ruby: use real traceback punctuation (backtick+apostrophe) instead of
  plain single quotes so LLM annotators can match actual Ruby output.
- Frames-vs-lines: clarify that a frame may span multiple source lines
  (Python emits two lines per frame) to prevent confusion with
  log_content's line-counting rule.
- Test brittleness: anchor dotnet_format test on 'File.cs:line 42' and
  cofire test on the full '"stack_trace": 1 AND "log_content": 1'
  string, since bare substrings '.NET' and 'fire BOTH' pre-existed in
  other parts of SYSTEM_PROMPT and would pass even if stack_trace were
  deleted.

Unified diff / git patch detection with strict required-marker rule: at
least one of (@@ hunk header, diff --git header, adjacent --- / +++ pair)
must be present. Line prefixes alone are insufficient — prevents markdown
bullet lists from dominating false positives at 0.1% class prevalence.

…ules

Two precision refinements to the diff_patch definition from code-quality
review, carrying forward the same tightening pattern applied to
log_content in Phase 3:

- 'adjacent lines' disambiguated inline to 'immediately consecutive lines
  with no blank line between them' — a small LLM could otherwise count
  ---/blank/+++ as adjacent. Phrase 'adjacent lines' retained for the
  existing regression-guard test.
- Email-patch header rule: replaced 'co-occurring' (ambiguous, same bug
  we caught in log_content's logfmt rule) with 'within the same
  contiguous block as'. Also labelled the literal 'Mon Sep 17 00:00:00
  2001' constant explicitly as a git-format-patch sentinel to prevent
  small models from treating it as a date placeholder.

Composes the iter16 5k audit sample: stratified sample via the existing
stratified_sample() helper, plus targeted injection of known-positive
candidates for log_content, stack_trace, and diff_patch. Injected rows
are tagged with an audit_source column so the downstream audit can
exclude them from the agreement metric and measure per-label recall on
them separately.

Two low-cost improvements to the error surface for the upcoming Phase 8
run on the real 90k corpus:

- Emit a WARNING on stderr when an injection-regex label matches zero
  rows. A silent missed label would otherwise look identical to a
  successfully-matched label in the final report.
- Validate that the input corpus has required 'sub_type' and 'text'
  columns and raise ValueError with a clear message at the entry point,
  rather than failing deep inside stratified_sample() or polars string
  ops with a confusing downstream error.

Computes per-label inter-annotator agreement on stratified rows
(injected positives excluded — they'd inflate the stat) and per-new-label
recall on injected positives using majority vote across 3 annotator
models. Emits a markdown report with pass/fail verdicts against the
two quantitative pass criteria from the iter16 spec.

Three defensive improvements to audit_semantic_labels.py before the
Phase 8 run on real annotated parquets:

- NaN handling in format_report: a NaN score (no data) now renders as
  'N/A (no data)' / 'N/A (no injected rows)' rather than silently
  comparing False against the threshold and producing a confusing FAIL.
  Missing recall data still counts as criterion-2 failure (can't certify
  recall we couldn't measure).
- Schema drift guards in compute_agreement_across_models: explicit
  ValueError if the 3 model parquets have mismatched det_* columns or
  row counts, rather than a cryptic downstream KeyError mid-loop.
- Cleanup: remove unused DETECTION_LABELS import from the module and
  unused tempfile/Path imports from the test file.

The 5k audit surfaced an ambiguity: CSV files with header columns
timestamp,level,message containing uppercase log severities were
annotated inconsistently (gemini fired, sonnet + gpt54mini did not).
The existing definition enumerated log schemas as nginx/syslog/JSON but
not CSV/TSV, and the anti-signal 'CSV/TSV with date columns' was too
broad — it correctly excluded sales CSVs with order_date columns but
also excluded CSV log exports.

Refinement: CSV/TSV log records fire log_content when BOTH a
timestamp-like column (timestamp/time/ts/date) AND a severity-like
column (level/severity/log_level) are present AND the severity values
are uppercase log levels. A CSV with only a date column still does
NOT fire.

Targeted verification on the 24 CSV log rows from the iter16 audit
under the refined prompt: 24/24 fire log_content unanimously across
gemini-3-flash, sonnet-4.6, and gpt-5.4-mini.

5k audit results and gate decision for the 3 semantic detection labels.
All three (log_content, stack_trace, diff_patch) have inter-annotator
agreement >=0.9986 on the stratified 4984-row cohort. Raw recall was
misleadingly low (62.5% stack_trace, 84% log_content) due to injection-
regex overmatch and a CSV log-records ambiguity; manual spot-check
identified both and neither reflected a prompt-quality problem. The CSV
ambiguity was fixed in c1ec175 with 24/24 unanimous post-fix recall on
affected rows. Gate: PASS. Proceed to next branch for full 90k run.

Restructures the iter16 Follow-Up section to make the A/B regression
audit a concrete, mandatory pre-gate on the 90k annotation run rather
than a dated 'if we want rigorous evidence' aside.

iter16's absolute-agreement audit has no control group and cannot prove
whether the 12 pre-existing labels below the 0.995 threshold (plain at
0.9391, markdown at 0.9507, etc.) reflect noise floors or silent
regressions from the longer SYSTEM_PROMPT. Before committing ~$400-600
on the 90k run, iter17 must:

1. Worktree off commit 22bc292 (last iter15 prompt state, Phase 0 tests
   already green)
2. Re-annotate the existing iter16_5k_input.parquet with the iter15
   prompt across the same 3 models
3. Compute per-label agreement delta
4. Gate: no pre-iter16 label may drop agreement by more than 0.005 vs
   iter15 baseline

Also clarifies the post-retrain 5k eval as the primary model-quality
gate (category/sub_type F1 regression check + per-detection-label F1)
and reorders operational follow-ups (python -m guard, injection regex
precision) to land before the A/B runs so the tooling is sound.